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SECTION  I 


INTRODUCTION 

A  decision  problem  can  be  considered  as  a  problem  in  classifying  objects 
according  to  their  characteristics.  The  following  are  simple  examples  of  deci¬ 
sions  to  be  made: 

Into  which  category  shall  a  book  be  placed  according  to  its  contents? 

Into  which  category  shall  the  outcome  of  a  statistical  test  be  placed 
depending  upon  the  numerical  result  (shall  the  hypothesis  be  accepted 
or  rejected)? 

Which  action  shall  be  taken  in  a  military  situation  described  by  certain 
characteristics? 

This  list  could  arbitrarily  be  extended  to  demonstrate  that  all  decision  problems 
are  basically  problems  of  classification.  For  decisions  under  risk  or  uncer¬ 
tainty,  some  finer  points  have  to  be  introduced,  but  the  basic  structure  of  the 
problem  remains  unchanged. 
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SECTION  II 


DISCUSSION 

The  characteristics  of  the  objects  to  be  classified  can  be  either  yes/no 
(binary  valued)  statements  of  possessing  or  not  possessing  a  certain  property, 
they  can  be  indications  of  which  of  a  discrete  set  (integral  valued)  of  possibil¬ 
ities  applies,  or  they  can  be  one  of  a  continuous  set  (real  valued)  which  describes 
properties  of  the  object  quantitatively.  For  the  different  properties  of  one 
object,  any  one  of  the  three  named  cases  may  be  applicable.  In  this  report  it 
is  assumed  that  each  property  of  the  object  is  given  by  a  real  number. 

The  least  restrictive  way  of  classification  is  to  have  a  complete  listing 
of  all  objects  with  an  indication  of  the  class  to  which  each  belongs.  In  most 
cases,  such  a  listing  would  be  extremely  long,  if  not  infinitely  long,  and 
therefore  impractical  or  impossible  to  use.  Classifications  are  rarely  given 
in  this  manner.  Usually,  objects  are  classified  according  to  functions  of  their 
parameters.  Therefore,  practical  decisions  generally  consist  of  evaluating 
certain  functions  of  the  characteristics;  namely  a  mapping  of  the  character¬ 
istics  space  into  a  decision  space. 

If  the  classification  is  given  by  a  complete  listing  of  objects,  any 
completely  arbitrary  classification  is  possible.  If  the  classification  is  done 
by  evaluating  "reasonable"  functions,  then  it  is  generally  the  case  that  classes 
correspond  in  a  1-1  manner  to  regions  in  the  decision  space.  Iii  many  cases, 
it  holds  that  with  any  two  objects  which  belong  to  one  dlass,  all  objects 
"between"  them  belong  to  the  same  class.  This  is  the  same  as  the  mathematical 
statement  that  the  classes  can  be  described  by  non-intersecting  convex  sets  in 
the  characteristics  space.  One  of  the  important  properties  of  convex  sets  is 
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that  any  two  of  them,  which  do  not  intersect,  can  be  separated  by  a  hyperplane. 

% 

Therefore,  we  can  separate  all  classes  of  these  objects  by  a  sufficient  number 
of  hyperplanes.  Separation  by  a  hyperplane  is  mathematically  equivaler  ^  ^ 

the  evaluation  of  a  linear  function,  which  is  the  type  of  function  most  easily 
handled  mathematically.  Therefore,  under  reasonable  restrictions,  a  deci¬ 
sion  problem  can  be  reduced  from  an  extensive  search  procedure,  through 
a  complete  listing  or  evaluation  of  arbitrary  functions,  to  the  evaluation  of 
linear  functions  which  are  easily  handled. 

The  best  means  of  determining  these  linear  functions  is  a  problem  which 
will  be  dealt  with  in  a  future  report.  This  present  report  is  concerned  with 
the  problem:  How  many  linear  functions  are  needed  to  represent  a  certain 
number  of  classes?  This  number  is  dependent  upon  the  specific  configuration 
of  the  convex  sets.  However,  it  is  worthwhile  to  know  at  least  upper  and 

lower  bounds  for  this  number.  • 

UPPER  LIMIT 

• 

The  upper  bound  is  easily  obtained,  K,  between  any  two  out  of  the  R 
regions,  a  separating  hyperplane  is  used,  then  all  classes  are  separated. 

Therefoi-e  this  number  is  (R/2),  The  lower  bound  is  more  difficult  to  obtain. 

LOWER  LIMIT 

If  the  maximum  number  of  regions  R(m,  n)  into  which  an  n-dimensional 
space  can  be  divided  by  m  h3^perplanes  can  be  determined,  then  this  would 
give  a  lower  limit  to  the  number  m  of  hyperplanes  required  to  separate 
R(m,  n'  regioiis  in  an  n-dimensional  space. 
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This  maximum  number  of  regions 


« 


« 


» 


R(m,  n) 


m  >  1,  n  5: 1  . 


(1) 


The  following  proof  uses  essentially  the  argument  of  Winder,  who  deter¬ 
mined  the  maximum  number  of  regions  N(m,  n)  obtained  by  m  hyperplanes 
in  an  n-dimensional  Euclidean  space  when  all  hyperplanes  intersect  in  one 
point  to  be 


N(m,  n) 


k=0 


The  same  result  has  been  obtained  by  Cameron. 


[2] 


(2) 


In  order  to  obtain  the  maximum  number  of  regions,  the  hyperplanes 
cannot  have  completely  arbitrary  positions.  It  is  defined  that  m  hyperplanes 
in  an  n-dimensional  Euclidean  space  are  in  independent  position,  if  every  sub¬ 
set  of  k  n  of  them  has  an  n-k  dimensional  linear  manifold  as  an  intersec¬ 
tion,  and  any  intersection  of  n+1  is  empty.  For  any  m  and  n,  hjperplanes 
exist  in  independent  position,  since  this  is  equivalent  to  the  statement  that 
there  are  sets  of  m  linear  equations  in  n  variables  such  that  all  subsets  of 
k  ^  n  equations  have  exactly  n-k  dimensional  solution  spaces  and  all  sub¬ 
sets  of  n+1  have  no  solution. 


♦Cameron  proves  only  that  the  maximum  number  cannot  be  larger  than  this 
expression,  because  he  counts  the  combinatorial  possiblities  without  regard 
to  the  possibilities  of  their  geometric  realization.  Winder  seems  to  be  aware 
of  this  point  as  his  definition  of  "general  position"  shows,  but  does  not  men¬ 
tion  the  problem  explicitly. 
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Theorem; 


The  maximum  number  of  regions  into  which  an  n-dimensional  Euclidean 
space  can  be  divided  by  m  hyperplanes  is 


R(m,  n) 


m 

r-n 


(3) 


♦ 


« 


and  this  number  is  obtained  by  m  hyperplanes  in  independent  position. 

Proof: 

Since  m  points  divide  a  line  into  m+1  parts,  R(m,  1)  =  m+1  which 
equals 


k=0 


One  hyperplane  divides  any  Euclidean  space  into  two  parts,  R(l,  n)  =  2, 
which  equals 


2. 


(5) 


Starting  with  these  values,  the  theorem  can  be  proved  by  induction. 

Assume  that  m-1  hyperplanes  in  independent  positions  are  given;  they  divide 

th 

the  space  into  the  maximal  number  of  R(m-1,  n)  regions.  Any  m  hyper¬ 
plane  intersects  at  most  all  the  other  m-1  hyperplanes,  and  if  all  m  are  in 

th 

independent  position,  then  the  m  one  intersects  all  the  other  m-1  in 
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(n-2)-(limensional  subspaces.  These  subspaces  are  Iwperphines  in  independent 
position  with  the  (n-l)-dimensional  space  given  by  the  hyperplane.  Each 
of  the  R(m-1,  n-1)  sections,  into  which  this  hyperplane  is  divided,  divides 
one  of  the  regions  (determined  by  the  m-1  hyperplanes)  in  the  n-dimensional 
space  into  two  regions,  adding  n-1)  to  the  previous  R(m-1,  n)  regions. 

The  result  is  the  recurring  formula 

R(m.  n)  =  R(m-1.  n)  +  R(m-l.  n-1).  (G) 

This  formula  has  exactly  the  same  structure  as  that  obtained  by  Winder  and 
Camei’on.  From  the  inductive  hypothesis 


R(m-1.  n) 


>  ("-■) 


k=0 


(7) 


n-1 


R(m-1 


and  the  identity 


■ =  y,  {V)  =  I  (t/) 

k=0  k=0 

(:)  ■  (r)  •  (";*) . 


tlie  lollowiiiff  is  obtained 


(8) 


(9) 


R(m-1 .  n)  +  R(m-1, 


k=0  ^  k=0 

(10) 


This  proves  out  the  theorem. 
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SOME  NUMERICAL  VALUES 


Table  I  contains  the  values  of  R(m,  n)  for  small  values  of  m  and  n. 
Table  n  contains  the  values  of  N(m,  n)  as  given  by  Cameron,  for  somi.  small 
values  of  m  and  n. 


Table  I;  R(m,  n) 


V 


* 

Table  II;  N(m,  n) 


8 


V 


The  values  above  the  main  diagonals  have  to  be  the  same  in  both  cases  since, 
for  no  more  than  n  hyperplanes  in  an  n-dimensional  space,  it  is  not  necessary 
that  all  hjTJerplanes  go  through  one  point. 


The  values  below  the  main  diagonal  fit  the  following  pattern; 


N(m,  n)  =  2R(m-l,  n-1) 


(11) 


which  is  explained  by  a  comparison  of  the  formulas  (1)  and  (2).  Furthermore, 


R(m,  n)  =  N(m,  n)  + 


(12) 


since  the  parts  of  the  schemes  below  the  diagonal  differ  by  a  column  and  diagonal 
of  one,  and  that  the  recursion  formula  (6)  is  that  of  Pascal's  triangle.  How¬ 
ever,  there  seems  to  be  no  simple  geometric  interpretation  of  both  these 
formulas. 

APPROXIMATION 

It  may  be  noticed  that  when  n  ^  m,  R(m,  n)  =  2^^,  On  the  other  hand, 
if  m  »  n  and  for  large  n  Cameron  has  developed  an  approximation  for 
N(m,  n)  which  can  be  used  to  obtain  a  similar  expression  for  R(m,  n).  t^der 
these  conditions,  Cameron  finds  that 


(13) 


and  since 


R(m,  n)  =  -^  N(m+1,  n+1) 


(14) 
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under  the  same  conditions 


R(m,  n) 


TT  (n+1) 


(15) 


H.  C.  Joksch 


^  /tA\ 


D.  M,  Liss 
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